Sentiment Analysis in the Context of Disaster Responses

“Requirements” for a Sentiment Analysis Methodology for Disaster Response

To create a reliable “pipeline” that can be meaningful in the applied context of a disaster response:

1. An appropriate methodology

   • Which is the best methodology for our use-cases? What characteristics are to be prioritised to fit the needs?

   • An experimentation phase will likely be required. That phase will include both method and dataset testing

   • Open quesion: Use what is “out there” and construct a pipeline approach for disaster datasets (from input CSV to output spaital analyis) or research model optimisation? Probably a question of resources.

2. Training datasets

   • Scarce availability of disaster-related datasets

   • Option of “rehydrating” tweet_ids, but less than 50% are actually retrievable

   • Depending on aims: multi-lingual? English? Multimodal?

   • Own dataset annotation?

3. A “standardised” spatial and temporal analysis method should be assembled

   • Hotspot analysis

   • Trends over time

4. Fast output generation (template-based?)

(A Few) Methods

Model comparisons should be expanded, but also purposefully chosen.

	Models	Precision	Recall	Accuracy	F1
0	SentiWordNet	0.50	0.50	0.48	0.46
1	VADER	0.67	0.64	0.64	0.64
2	Naive Bayes	0.80	0.79	0.79	0.79
3	SVM	0.88	0.88	0.88	0.88
4	RoBERTa	0.71	0.73	0.71	0.71
5	BERTweet	0.71	0.72	0.70	0.71
6	Fine Tuned BERT	0.78	0.78	0.78	0.78

Datasets

A start…

#	Name	Size	Polarity	Twitter Data	Link
1	Sentiment140	1.6m	neg, neu, pos	yes	link
2	Twitter US Airline Sentiment	55K	neg, neu, pos	yes	link
3	SemEval-2015 Task 10: Sentiment Analysis in Twitter	3097	neg, neu, pos	yes	link
4	SemEval-2016 Task 4: Sentiment Analysis in Twitter	10K	neg, neu, pos	yes	link
4	SemEval-2016 Task 5: Aspect-Based Sentiment Analysis	10K	entity, attribute, pos, neu, neg, target	yes	link
5	SemEval-2017 Task 4: Sentiment Analysis in Twitter		neg, neu, pos	yes	link
6	SemEval-2018 Task 1: Affect in Tweets	7K	joy, fear, anger, sadness	yes	link
7	The Sanders Twitter Corpus	5K	neg, neu, pos, irrelevant	yes	link
8	The CrisisNLP dataset	x million (various use cases)	diverse	yes	link
9	TBCOV: Two Billion Multilingual COVID-19 Tweets with Sentiment, Entity, Geo, and Gender Labels	2 b	neg, neu, pos	yes	link
10	Natural Hazards Twitter Dataset	40K	pos, neg	yes	link
11	The TweetEval dataset
12	IMDb movie reviews dataset
13	Stanford Sentiment Treebank

---

In everyday life…

"Been trying to clear out my left ear with peroxide for the last 3 hours. This shit is bananas. I hate not being able to hear"

"Manchester United confirms that Carlos Tevez is leaving the club... so sad"

I wanna do something fun but have no clue what to do. Tre sleep & @krob5858 is taking a nap. So its just me

[Twitter Sentiment dataset with 30K tweets: see dataest]

	text	label	sentiment
2130	just woke up, I`m starving	0	neutral
11639	four shots of novacaine in my mouth my right cheek is totally numb, booo.	-1	negative
15569	Who`d have thought Wallace & Gromit and the team behind Monkey Island could have combined so disastrously?	-1	negative
8148	Don`t know yet Lemme know if you come up with something though.	0	neutral
18341	flu or allergy??? ... Doesn`t matter, just try to squeeze my Sundayyyy	-1	negative

---

In a disaster…

"Hope the people there are in a safe location and help reaches those in need during Hurricane Harvey. #prayfortexas"

"And the president pardons him on Friday during the prime of tropical storm Harvey."

"Our View: Hurricane Harvey proves we can still save each other"

"Still looking for an answer for power outages due to hurricane Harvey. I can have the best internet connection but no power = no sling"

There’s a scarcity of sentiment labelled twitter datasets. This one is derived from The Climate Change Twitter Dataset:

	date	text
47598	Mon Aug 28 23:29:00 +0000 2017	Hurricane Harvey leaves Texans without internet, phone service http://ift.tt/2xHXy6A�
32134	Sat Aug 26 00:51:00 +0000 2017	On a day when the nation should be focusing on Hurricane Harvey, Trump decides to pardon a racist, convicted felon.Actions-Trump is racist!
3945	Fri Aug 25 02:44:00 +0000 2017	Check out the latest on Hurricane Harvey and our Friday Football Forecast. https://www.41nbc.com/2017/08/24/harvey-threatens-texas-nice-weekend-store-middle-ga/��
26719	Sat Aug 26 00:25:00 +0000 2017	I added a video to a @YouTube playlist http://youtu.be/Y8as4cuFZ0A?a� LIVE COVERAGE: EXTREMELY DANGEROUS Category 4 Hurricane Harvey BATTERS
13588	Fri Aug 25 12:47:00 +0000 2017	Food safety tips for Hurricane Harvey http://www.chron.com/news/houston-weather/hurricanes/article/Food-safety-tips-for-Hurricane-Harvey-Houston-11956465.php?cmpid=twitter-mobile�� via @houstonchron

Some pos, neg datasets however are readily available. These are a few examples from the [Natural Hazards Twitter Dataset]('https://github.com/Dong-UTIL/Natural-Hazards-Twitter-Dataset'):

2017 Hurricane Harvey unlabelled dataset: 30000 tweets
2017 Hurricane Harvey labelled dataset (pos, neg): 7823 tweets
2018 Wildfires in the US labelled dataset (pos, neg): 4596 tweets
2013 floods in the US labelled dataset (pos, neg): 3597 tweets

Lexicon-Based Methods

Lexicon-based approaches are still used in certain cases, especially when the goal is to perform sentiment analysis quickly and with a limited amount of computational resources. In lexicon-based sentiment analysis, words are assigned a sentiment score based on a pre-existing lexicon, such as SentiWordNet or the AFINN lexicon, and the overall sentiment of a text is calculated by summing the sentiment scores of the individual words.

Lexicon-based approaches have several advantages:

• Ease of use

• Speed

• Interpretability

But there are downsides:

• Difficulties with sarcasm, negation, sentiment of words in context

	Sentiment Score
beautiful	2.9
beautifuler	2.1
beautifulest	2.6
beautifully	2.7
beautifulness	2.6
beautify	2.3
beautifying	2.3
beauts	1.7
beauty	2.8
belittle	-1.9

---

Lexicon: SentiWordNet

SentiWordNet is built via a semi supervised method and could be a valuable resource for performing opinion mining tasks: it provides a readily available database of term sentiment information for the English language, and could be used as a replacement to the process of manually deriving ad-hoc opinion lexicons.

by nltk (see model info)

import nltk
nltk.download('sentiwordnet')
from nltk.corpus import sentiwordnet as swn
from nltk.tokenize import word_tokenize

def get_sentiment(tweet):
    words = word_tokenize(tweet)
    sentiment = 0.0
    for word in words:
        synset = list(swn.senti_synsets(word))
        if len(synset) > 0:
            sentiment += synset[0].pos_score() - synset[0].neg_score()
    return sentiment

def classify_sentiment(tweet):
    sentiment = get_sentiment(tweet)
    if sentiment > 0:
        return 1
    elif sentiment < 0:
        return -1
    else:
        return 0

# normal tweets for evaluation and comparison
sentiments1 = [classify_sentiment(tweet) for tweet in normal_tweets['text']]
normal_tweets['nltk_sentiwordnet'] = sentiments1

# disaster tweets for comparison
sentiments2 = [classify_sentiment(tweet) for tweet in harvey_data['text']]
harvey_data['nltk_sentiwordnet'] = sentiments2

	Precision	Recall	Accuracy	F1
SentiWordNet	0.5	0.5	0.48	0.46

---

Lexicon: VADER (Valence Aware Dictionary and sEntiment Reasoner)

VADER is specifically attuned to sentiments expressed in social media. It uses a combination of sentiment-related words and emojis, along with some simple heuristics (punctuation, capitalisation, degree modifiers, conjuctions), to assign a sentiment score (positive, negative, or neutral) to a given piece of text. It’s output sentiment score is a numeric score between -1 and +1. The word sentiment scores range from -4 to 4 (neg to pos).

by Hutto & Gilbert see model info

from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer    # VADER
sa = SentimentIntensityAnalyzer()  

scores = {}
listscores = []

for tweet in normal_tweets['text']:
    scores[sa.polarity_scores(tweet)['compound'] ] = str(tweet)
    listscores.append(sa.polarity_scores(tweet)['compound'])

normal_tweets['vader'] = listscores

scores1 = {}
listscores1 = []

for tweet in harvey_data['text']:
    scores1[sa.polarity_scores(tweet)['compound'] ] = str(tweet)
    listscores1.append(sa.polarity_scores(tweet)['compound'])

harvey_data['vader'] = listscores1

normal_tweets['vader'] = [1 if value >= 0.3 else -1 if value <= -0.3 else 0 for value in normal_tweets['vader']]
harvey_data['vader'] = [1 if value >= 0.3 else -1 if value <= -0.3 else 0 for value in harvey_data['vader']]

performance = calc_error_matrices(normal_tweets, 'VADER', 'label', 'vader')
performance

	Precision	Recall	Accuracy	F1
SentiWordNet	0.50	0.50	0.48	0.46
VADER	0.67	0.64	0.64	0.64

---

Machine Learning Method: Naive Bayes

Naive Bayes is a probabilistic algorithm (based on Bayes’ theorem). It uses the probability of words or terms appearing in documents of different categories to determine the likelihood that a new document belongs to each category. It involves two basic steps:

Training: The algorithm learns the probability of words or terms appearing in each category. This is simply done by counting the number of occurrences of each word or term in the training corpus for each category and then computing the probability of each word or term given the category. This results in a set of word probabilities for each category.

Classification: The algorithm uses the probabilities learned in the training step to classify new documents. For a new document, the algorithm calculates the likelihood of the document being in each category based on the probabilities of its words or terms. The category with the highest likelihood is chosen as the classification for the document.

The “naive” aspect of Naive Bayes comes from the assumption that the occurrences of words or terms in a document are independent of one another, which is not always true(!).

by nltk see model info

from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.metrics import accuracy_score

# split the data into training and testing sets
train_text, test_text, train_labels, test_labels = train_test_split(normal_tweets['text'], normal_tweets['label'], random_state=42)

# create a bag of words representation of the text data
vectorizer = CountVectorizer()
train_text_vectors = vectorizer.fit_transform(train_text)
test_text_vectors = vectorizer.transform(test_text)

# fit a Naive Bayes classifier on the training data
clf = MultinomialNB()
clf.fit(train_text_vectors, train_labels)

text_vectors = vectorizer.transform(normal_tweets['text'])

# make predictions on the text data using the trained classifier
predictions = clf.predict(text_vectors)

# add the predictions as a new column in the dataframe
normal_tweets['naive_bayes'] = predictions

# same for disaster data
text_vectors1 = vectorizer.transform(harvey_data['text'])
predictions1 = clf.predict(text_vectors1)

# add the predictions as a new column in the dataframe
harvey_data['naive_bayes'] = predictions1

plotting('naive_bayes', 'Naive Bayes')

	Precision	Recall	Accuracy	F1
SentiWordNet	0.50	0.50	0.48	0.46
VADER	0.67	0.64	0.64	0.64
Naive Bayes	0.80	0.79	0.79	0.79

---

Machine Learning Method: Support Vector Machine

Using readily available libraries like sklearn, an SVM classifier (in this case SVC (Support Vector Classification)) can be trained for binary classification problems like pos-neg sentiment classification. The SVC classifier predicts the class label of a given sample based on the feature set. It solves the optimization problem to find a hyperplane that maximally separates the positive and negative class samples in the feature space. The optimization problem is solved using the maximum margin principle, where the margin is the distance between the hyperplane and the closest samples from each class, called support vectors. The SVC classifier is a useful tool for performing binary classification problems, particularly when the number of features is high, and the data is not linearly separable.

by sklearn see model info

# Import libraries
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
from sklearn import svm

# split the data into training and testing sets
train_text, test_text, train_labels, test_labels = train_test_split(normal_tweets['text'], normal_tweets['label'], random_state=42)

# create a bag of words representation of the text data
vectorizer = CountVectorizer()
train_text_vectors = vectorizer.fit_transform(train_text)
test_text_vectors = vectorizer.transform(test_text)

# fit an SVM classifier on the training data
clf = SVC(kernel='linear')
clf.fit(train_text_vectors, train_labels)

# Test the classifier
X_new = vectorizer.transform(normal_tweets["text"])
y_pred = clf.predict(X_new)
normal_tweets['svm'] = y_pred

X_new1 = vectorizer.transform(harvey_data["text"])
y_pred1 = clf.predict(X_new1)
harvey_data['svm'] = y_pred1

performance = calc_error_matrices(normal_tweets, 'SVM', 'label', 'svm')
performance

	Precision	Recall	Accuracy	F1
SentiWordNet	0.50	0.50	0.48	0.46
VADER	0.67	0.64	0.64	0.64
Naive Bayes	0.80	0.79	0.79	0.79
SVM	0.88	0.88	0.88	0.88

---

Deep Learning Models

In recent years, the use of lexicon-based methods and traditional machine learning models for sentiment analysis has decreased in popularity. This shift has been largely driven by the emergence of deep learning models, which have shown impressive results on NLP tasks such as sentiment classification. With the availability of vast amounts of text data, deep learning models such as transformers have been able to capture more complex patterns and relationships within the data, leading to improved performance compared to lexicon-based and machine learning models. As a result, deep learning methods are now seen as the state-of-the-art approach for sentiment analysis and other NLP tasks.

---

Recurrent Neural Network

A Recurrent Neural Network is a generalization of feedforward neural network that has an “internal memory”. An RNN is recurrent in nature as it performs the same function for every input of data while the output of the current input depends on the past one computation. After producing the output, it is copied and sent back into the recurrent network. For making a decision, it considers the current input and the output that it has learned from the previous input.

see model info

model = ltsm_model((maxLen,), word_to_vec_map, word_to_index)
model.summary()

Model: "model"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_2 (InputLayer)        [(None, 29)]              0         
                                                                 
 embedding_1 (Embedding)     (None, 29, 50)            20000050  
                                                                 
 lstm (LSTM)                 (None, 29, 128)           91648     
                                                                 
 dropout (Dropout)           (None, 29, 128)           0         
                                                                 
 lstm_1 (LSTM)               (None, 128)               131584    
                                                                 
 dropout_1 (Dropout)         (None, 128)               0         
                                                                 
 dense_1 (Dense)             (None, 3)                 387       
                                                                 
 activation (Activation)     (None, 3)                 0         
                                                                 
=================================================================
Total params: 20,223,669
Trainable params: 223,619
Non-trainable params: 20,000,050
_________________________________________________________________

---

Gated Recurrent Units

A Gated Recurrent Unit (GRU) is a type of Recurrent Neural Network (RNN) that is used to process sequential data. It is designed to handle the vanishing gradient problem that can occur when training traditional RNNs by incorporating gating mechanisms.

see model info

# Defining GRU model
gru_model = Sequential(name="GRU_Model")
gru_model.add(Embedding(vocab_size,
                        embd_len,
                        input_length=max_words))
gru_model.add(GRU(128,
                  activation='tanh',
                  return_sequences=False))
gru_model.add(Dense(1, activation='sigmoid'))
 
# Printing the Summary
print(gru_model.summary())
 
# Compiling the model
gru_model.compile(
    loss="binary_crossentropy",
    optimizer='adam',
    metrics=['accuracy']
)
 
# Training the GRU model
history2 = gru_model.fit(x_train_, y_train_,
                         batch_size=64,
                         epochs=5,
                         verbose=1,
                         validation_data=(x_valid, y_valid))
 
# Printing model score on test data
print()
print("GRU model Score---> ", gru_model.evaluate(x_test, y_test, verbose=0))

---

Long Short Term Memory Network

LSTM (Long Short-Term Memory) is a type of Recurrent Neural Network (RNN) that is designed to overcome the problem of vanishing gradients in traditional RNNs by introducing a gating mechanism that controls the flow of information within the network. An LSTM network contains a series of memory cells that are responsible for remembering the information from earlier time steps. There are three gates in an LSTM cell that control the flow of information: the input gate, the forget gate, and the output gate. These gates allow the LSTM to selectively decide which information to forget and which information to pass to the output.

see model info

# Defining LSTM model
lstm_model = Sequential(name="LSTM_Model")
lstm_model.add(Embedding(vocab_size,
                         embd_len,
                         input_length=max_words))
lstm_model.add(LSTM(128,
                    activation='relu',
                    return_sequences=False))
lstm_model.add(Dense(1, activation='sigmoid'))
 
# Printing Model Summary
print(lstm_model.summary())
 
# Compiling the model
lstm_model.compile(
    loss="binary_crossentropy",
    optimizer='adam',
    metrics=['accuracy']
)
 
# Training the model
history3 = lstm_model.fit(x_train_, y_train_,
                          batch_size=64,
                          epochs=5,
                          verbose=2,
                          validation_data=(x_valid, y_valid))
 
# Displaying the model accuracy on test data
print()
print("LSTM model Score---> ", lstm_model.evaluate(x_test, y_test, verbose=0))

---

Transformer-Based Models

In recent years, the transformer model has revolutionized the field of NLP. This ‘new’ deep learning approach has been highly successful in a variety NLP tasks, including sentiment analysis. The transformer model offers several advantages over traditional machine learning and even other deep learning approaches and have been shown to outperform traditional machine learning and other deep learning methods on NLP tasks, particularly sentiment analysis. Some of the key advantages it has are:

• The encoder-decoder framework: Encoder generates a representation of the input (semantic, context, positional) and the decoder generates output. Common use case: sequence to sequence translation tasks.

• Attention mechanisms: Deals with the information bottleneck of the traditional encoder-decoder architecture (where one final encoder hidden state is passed to decoder) by allowing the decoder to access the hidden states at each step and being able to prioritise which state is most relevant.

• Transfer learning (i.e. fine-tuning a pre-trained language model)

---

Transformer-Based Model: RoBERTa

RoBERTA (Robustly Optimized BERT Pretraining Approach) has the same architecture as BERT but marks an improved version of BERT for several reasons:

• RoBERTa was trained on 10x as much data as was used for BERT training (160GB, compared to 16GB for BERT)

• Dynamic masking was used during training, rather than fixed masking in BERT

• the next sentence prediction was left out during training, which is arguably not essential especially when considering tweets.

by cardiffnlp see model info

# apply in the form of a function so it can be called for usecase later on
def robertabase_apply(dataset):
    
    # create variable for labels (good to bad)
    labels= ['positive', 'neutral', 'negative']
    
    # lists to be filled
    cardiffroberta_sentiment_prediction = []
    cardiffroberta_sentiment_prediction_softmax = []
    cardiffroberta_sentiment_prediction_num = []
    
    # iterate over dataset
    for index, row in dataset.iterrows():
        text = row['text']
        text = preprocess(text)
        encoded_input = tokenizer_tw_rob_base_sent_lat(text, truncation=True, max_length=500, return_tensors='pt')
        output = model_tw_rob_base_sent_lat(**encoded_input)
        score = np.round(softmax(output[0][0].detach().numpy()), 4)
        label = config_tw_rob_base_sent_lat.id2label[np.argsort(score)[::-1][0]]
        cardiffroberta_sentiment_prediction.append(label)
        cardiffroberta_sentiment_prediction_softmax.append(max(score))
        # positive label
        if label == labels[0]:
            cardiffroberta_sentiment_prediction_num.append(1)
        # negative label
        elif label == labels[2]:
            cardiffroberta_sentiment_prediction_num.append(-1)
        # neutral label
        else:
            cardiffroberta_sentiment_prediction_num.append(0)


    #dataset['cardiffroberta_sentiment_prediction'] = cardiffroberta_sentiment_prediction
    #dataset['cardiffroberta_sentiment_prediction_softmax'] = cardiffroberta_sentiment_prediction_softmax
    dataset['cardiffroberta'] = cardiffroberta_sentiment_prediction_num

    model_name = "cardiffroberta"
    
    # model name and labels will be needed later on as input variables for plotting and mapping
    print("Variables that will later be required for plotting and mapping:")
    return model_name, labels

	Precision	Recall	Accuracy	F1
SentiWordNet	0.50	0.50	0.48	0.46
VADER	0.67	0.64	0.64	0.64
Naive Bayes	0.80	0.79	0.79	0.79
SVM	0.88	0.88	0.88	0.88
RoBERTa	0.71	0.73	0.71	0.71

---

Transformer-based Model: BERTweet

This is a BERTweet-base RoBERTa model trained on SemEval 2017 (~40k Tweets). It uses POS, NEG, NEU labels and is suitable for English and Spanish languages. pysentimiento is an open-source library for non-commercial use and scientific research purposes only. Please be aware that models are trained with third-party datasets and are subject to their respective licenses.

by finiteautomata see model info

# apply in the form of a function so it can be called for usecase later on
def bertweet_apply(dataset):
    
    # create labels variable
    labels = ['POS', 'NEU', 'NEG']
    
    # lists to be filled
    bertweet_sentiment_prediction = []
    bertweet_sentiment_prediction_softmax = []
    bertweet_sentiment_prediction_num = []  
    
    # iterate over dataframe
    for index, row in dataset.iterrows():
        text = row['text']
        text = bertweetpreprocess(text)
        result = bertweetanalyzer.predict(text)
        label = result.output
        bertweet_sentiment_prediction.append(label)
        bertweet_sentiment_prediction_softmax.append(np.round(result.probas[label], 4))
        if label == labels[0]:
            bertweet_sentiment_prediction_num.append(1)
        elif label == labels[2]:
            bertweet_sentiment_prediction_num.append(-1)
        else:
            bertweet_sentiment_prediction_num.append(0)


    #dataset['bertweet_sentiment_prediction'] = bertweet_sentiment_prediction
    #dataset['bertweet_sentiment_prediction_softmax'] = bertweet_sentiment_prediction_softmax
    dataset['bertweet'] = bertweet_sentiment_prediction_num
    
    model_name = "bertweet"
    
    # model name and labels will be needed later on as input variables for plotting and mapping
    print("Variables that will later be required for plotting and mapping:")    
    return model_name, labels

	Precision	Recall	Accuracy	F1
SentiWordNet	0.50	0.50	0.48	0.46
VADER	0.67	0.64	0.64	0.64
Naive Bayes	0.80	0.79	0.79	0.79
SVM	0.88	0.88	0.88	0.88
RoBERTa	0.71	0.73	0.71	0.71
BERTweet	0.71	0.72	0.70	0.71

---

Fine-Tuned Downstream Sentiment Analysis

This is a BERT base model (uncased), pretrained on English language using a masked language modeling (MLM) objective. This model is uncased: it does not make a difference between english and English.

This is a fine-tuned downstream version of the bert-base-uncased model for sentiment analysis, this model is not intended for further downstream fine-tuning for any other tasks. This model is trained on a classified dataset for text classification.

by seethal see model info

# apply in the form of a function so it can be called for usecase later on
def seethal_gen_data_apply(dataset):
    
    # create variable for labels (good to bad)
    labels= ['positive', 'neutral', 'negative']
    
    # lists to be filled
    seethal_gen_data_sentiment_prediction = []
    seethal_gen_data_sentiment_prediction_softmax = []
    seethal_gen_data_sentiment_prediction_num = []
    
    # iterate over dataset
    for index, row in dataset.iterrows():
        text = row['text']
        text = preprocess(text)
        encoded_input = tokenizer_seethal_gen_data(text, truncation=True, max_length=500, return_tensors='pt')
        output = model_seethal_gen_data(**encoded_input)
        score = np.round(softmax(output[0][0].detach().numpy()), 4)
        label = config_tw_rob_base_sent_lat.id2label[np.argsort(score)[::-1][0]]
        seethal_gen_data_sentiment_prediction.append(label)
        seethal_gen_data_sentiment_prediction_softmax.append(max(score))
        # positive label
        if label == labels[0]:
            seethal_gen_data_sentiment_prediction_num.append(1)
        # negative label
        elif label == labels[2]:
            seethal_gen_data_sentiment_prediction_num.append(-1)
        # neutral label
        else:
            seethal_gen_data_sentiment_prediction_num.append(0)


    #dataset['seethal_gen_data_sentiment_prediction'] = seethal_gen_data_sentiment_prediction
    #dataset['seethal_gen_data_sentiment_prediction_softmax'] = seethal_gen_data_sentiment_prediction_softmax
    dataset['finetunedBERT'] = seethal_gen_data_sentiment_prediction_num

    model_name = "seethal_gen_data"
    
    # model name and labels will be needed later on as input variables for plotting and mapping
    print("Variables that will later be required for plotting and mapping:")
    return model_name, labels

	Precision	Recall	Accuracy	F1
SentiWordNet	0.50	0.50	0.48	0.46
VADER	0.67	0.64	0.64	0.64
Naive Bayes	0.80	0.79	0.79	0.79
SVM	0.88	0.88	0.88	0.88
RoBERTa	0.71	0.73	0.71	0.71
BERTweet	0.71	0.72	0.70	0.71
Fine Tuned BERT	0.78	0.78	0.78	0.78

---

Data Visualisations

gdf_harvey.head()

	TWEET_ID	DATE	text	LAT	LONG	MATCHING_I	TOPIC_NUMB	PROBABILIT	WEIGHT	geometry	bertweet
0	9.058685e+17	2017-09-07 19:00:49	Two more days!!! https://t.co/vydsOgG5RY	30.223722	-97.744334	29	0	0.066667	1	POINT (620838.338 3344242.541)	1
1	9.041387e+17	2017-09-03 00:27:26	Baton Rouge: 7:24pm: sunset	30.440000	-91.190000	212	0	0.666367	1	POINT (1251048.695 3393577.497)	0
2	9.057606e+17	2017-09-07 11:52:06	Be Prepared! Stay Prepared! @ Fire Station 93 ...	29.590136	-95.180497	17	8	0.666367	1	POINT (870027.953 3279466.422)	1
3	9.010665e+17	2017-08-25 12:59:26	#hurricaneharvey @ Houston, Texas https://t.co...	29.762900	-95.383200	1972	1	0.999534	1	POINT (849776.868 3297998.151)	0
4	9.058019e+17	2017-09-07 14:36:20	Just posted a photo @ Ruff Cuts & PurrFect...	30.228525	-92.658851	20	1	0.999650	1	POINT (1110801.489 3361168.665)	1

Ratios between pos, neu, and neg sentiments: [4891, 21570, 2472]

Country USA was either not found or too many matches found in centroids dataframe. Defaulting to USA.

Make this Notebook Trusted to load map: File -> Trust Notebook